Dynamic Joint Action Perception for Q-Learning Agents

Q-Iearning is a reinforcement learning alg()rithm that learns expected utilities for stateaction transitions through successive interactions with the environment The algorithm '5 simplicity as well as its convergence properties have made it a popular algorithm for study However; its non-parametric representation of utilities limits its effectiveness in environments with large amounts of perceptual input For example; in multiagent systems; each agent may need to consider the action selections of its counterparts in order to learn effective behaviors This creates a joint action space which grows exponentially with the number of agents in the system In such situations; the Q-learning algorithm quickly becomes intractable This paper presents a new algorithm; Dynamic ,Joint Action Perception; which addresses this problem by allowing each agent to dynamically perceive only those joint action distinctions which are relevant to its own payoffs The result is a smaller joint action space and improved scalability of Q-learning to systems with many agents